Optimizing data stream processing for large‐scale applications
نویسندگان
چکیده
منابع مشابه
COLA: Optimizing Stream Processing Applications via Graph Partitioning
In this paper, we describe an optimization scheme for fusing compile-time operators into reasonably-sized run-time software units called processing elements (PEs). Such PEs are the basic deployable units in System S, a highly scalable distributed stream processing middleware system. Finding a high quality fusion significantly benefits the performance of streaming jobs. In order to maximize thro...
متن کاملDesign principles for developing stream processing applications
Stream processing applications are used to ingest, process, and analyze continuous data streams from heterogeneous sources of live and stored data, generating streams of output results. These applications are, in many cases, complex, large-scale, low-latency, and distributed in nature. In this paper, we describe the design principles and architectural underpinnings for stream processing applica...
متن کاملVisual Debugging for Stream Processing Applications
Stream processing is a new computing paradigm that enables continuous and fast analysis of massive volumes of streaming data. Debugging streaming applications is not trivial, since they are typically distributed across multiple nodes and handle large amounts of data. Traditional debugging techniques like breakpoints often rely on a stop-the-world approach, which may be useful for debugging sing...
متن کاملOptimizing The Lazy DFA Approach for XML Stream Processing
Lazy DFA (Deterministic Finite Automata) approach has been recently proposed to for efficient XML stream data processing. This paper discusses the drawbacks of the approach, suggests several optimizations as solutions, and presents a detailed analysis for the processing model. The experiments show that our proposed approach is indeed effective and scalable.
متن کاملARES: an Adaptively Re-optimizing Engine for Stream Query Processing
Applications dealing with continuous streaming data such as sensor data processing, network traffic engineering, network monitoring, intrusion detection, financial monitoring etc are becoming more and more predominant. Such applications have to deal with multiple continuous data streams with inputs arriving at highly variable and unpredictable rates from various sources. These applications have...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Software: Practice and Experience
سال: 2018
ISSN: 0038-0644,1097-024X
DOI: 10.1002/spe.2596